Communications base station with decision function for distributing traffic across multiple backhauls

ABSTRACT

A communication station, such as a base station or access point, has multiple backhaul options and distributes backhaul data between the available backhaul options. The communication station includes a transceiver for transmitting and receiving data with user equipments, multiple backhaul interface modules, and a backhaul distribution module arranged for monitoring demand for backhaul bandwidth and distributing data over the backhauls based on the demand for backhaul bandwidth. Additional modules for user data and control plane processing may be included with the user/control distinction used in distributing data over the backhauls. The backhaul options may include a preferred backhaul and an alternate backhaul. Distributing data over the backhauls may be based, for example, on applications associated with the data, financial cost, delay, robustness, computational resources, and/or additional security associated with using a particular backhaul.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.61/505,262, filed Jul. 7, 2011, which is hereby incorporated byreference.

BACKGROUND

The present invention relates to the field of communication systems andto systems and methods for distributing traffic across multiplebackhauls using associated decision criteria.

In a communication network, such as a broadband wireless network, eachbase station or similar node is connected to the core network via abackhaul connection. Backhaul connections can be implemented using avariety of technologies such as point-to-point (PtP) wireless in avariety of frequencies both licensed and unlicensed, point-to-multipoint(PmP) wireless in a variety of frequencies, Ethernet over copper orfiber optic cable, cable modem, etc. These backhaul technologies canhave a variety of capacities, for instance 10 megabits per second(Mbps), 100 Mbps, and 1 gigabit per second are common capacities forEthernet technologies. Additionally, the backhaul choices may havedifference performance or operational costs. For instance one backhaulchoice may be owned by the operator of the base station while anothermay be leased. One backhaul choice may be more or less costly to installor maintain than other choices. Different backhaul choices may havedifferent robustness, such as unlicensed spectrum wireless versuslicensed spectrum wireless. Different backhaul choices may lead toadditional computational resources being used in the base station suchas the need to implement IP-SEC or TR-0069 tunneling over an untrustedbackhaul such as a backhaul not owned by the operator of the basestation.

SUMMARY

Systems and methods for distributing traffic across multiple backhaulconnections to and from a base station are provided. In one aspect, theinvention provides a communication station, including: a transceiverarranged for transmitting and receiving data with subscriber stations; aplurality of backhaul interface modules, each backhaul interface modulebeing arranged for providing backhaul communications over acorresponding one of a plurality of backhaul connections; a dataprocessing module for detecting information about applicationsassociated with the data transmitted and received by the transceiver;and a backhaul distribution module coupled to the plurality of backhaulinterface modules and arranged for monitoring demand for backhaulbandwidth and distributing data over the plurality of backhaulconnections based on the demand for backhaul bandwidth and the detectedinformation about the applications associated with the data.

In another aspect, the invention provides a method for use with acommunication station operable to communicate with user equipments via atransceiver and communicate with a network over a plurality of backhaulconnections. The method includes: monitoring demand for backhaulbandwidth; distributing backhaul data to a preferred one of theplurality of backhaul connections; detecting information aboutapplications associated with the backhaul data; determining whether thedemand for backhaul bandwidth exceeds the capacity of the preferred oneof the plurality of backhaul connections backhaul; and distributing atleast some of the backhaul data to an alternate one of the plurality ofbackhaul connections when the demand for backhaul bandwidth exceeds thecapacity of the preferred one of the plurality of backhaul connections,the backhaul data distributed to an alternate one of the plurality ofbackhaul connections being selected utilizing the detected informationabout the applications associated with the backhaul data.

Other features and advantages of the present invention should beapparent from the following description which illustrates, by way ofexample, aspects of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its structure andoperation, may be gleaned in part by study of the accompanying drawings,in which like reference numerals refer to like parts, and in which:

FIG. 1 is a block diagram of a wireless communication network in whichthe systems and methods disclosed herein can be implemented according toan embodiment;

FIG. 2 is block diagram of another wireless communication network inwhich the systems and methods disclosed herein can be implementedaccording to an embodiment;

FIG. 3 is a block diagram of a subset of the wireless communicationsystem of FIG. 1 in which a base station has multiple backhaul optionsavailable;

FIG. 4 is a functional block diagram of a station according to anembodiment;

FIG. 5 is a functional block diagram of an LTE picocell with multiplebackhauls; and

FIG. 6 is a flow diagram for a method for determining when to use analternate backhaul.

DETAILED DESCRIPTION

A base station with multiple backhaul choices which uses a method with adecision function, such as a cost function, to distribute backhaultraffic across the multiple backhaul choices is provided. The basestation disclosed herein can be used with backhaul technology thatconnects the base station to the core network for the purpose oftransporting either user plane data or control plane data or both.Embodiments have two or more backhaul choices available to them. Someembodiments have a primary or preferred backhaul choice and one or morealternative backhaul choices. Some embodiments use a decision functionto determine when to use one backhaul choice preferentially to anotheror when to use an alternate backhaul choice in addition to a preferredbackhaul choice. Some embodiments use a decision function to determinethe quantity of data to transport over a preferred backhaul choice andan alternate backhaul choice. Some embodiments classify data trafficinto groupings that are better suited to be transported over a preferredor an alternative backhaul. Some embodiments detect information aboutapplications associated with the data traffic and use the applicationinformation in the backhaul decision function. A base station may, forexample, inspect packets to determine application classes and/orspecific applications associated with the packets.

The decision function may have many embodiments dependent, for example,upon the available backhaul choices. Different backhaul options may havedifferent capacities. Different backhaul options may have differentlevels of robustness causing one option to be more likely to transportdata error free. Different backhaul options may have differentcomputational costs. For instance, one backhaul choice may be untrustedand require additional security procedures, such as IP-SEC, to avoidsecurity breaches. This additional security may require additionalcomputational resources. A base station may not have sufficientcomputational resources to apply the enhanced security to the peak datarate and, therefore must analyze the resource cost of using the backhaulwhich requires the extra security. Different backhaul choices may usediffering amounts of other resources, for example, electricity.Different backhaul choices may have different rate tariffs, for instanceif two different backhaul choices are owned by different operators.Different backhaul choices may have different delay characteristics orother differing performance characteristics that may be used in a costfunction.

The aforementioned inputs to the decision function may be static,dynamic or some combination of the two. The robustness of a backhauloption may vary over time due to the conditions of the backhaultransmission channel. The performance characteristics of a backhauloption such as capacity and delay may vary over time due to, forexample, the current usage of this backhaul option by other systems. Therate tariffs may also vary over time due to the specific rate tablesimplemented by a backhaul service provider.

The systems and methods disclosed herein can be applied to base stationsor similar nodes, such as a cable head end, in various communicationsystems, including but not limited to wireline and wirelesstechnologies. For example, the systems and methods disclosed herein canbe used with Cellular 2G, 3G, 4G (including Long Term Evolution (“LTE”),LTE Advanced, WiMax), WiFi, Ultra Mobile Broadband (“UMB”), cable modem,and other wireline or wireless technologies. Although the phrases andterms used herein to describe specific embodiments can be applied to aparticular technology or standard, the systems and methods describedherein are not limited to these specific standards.

FIG. 1 is a block diagram of a wireless communication network in whichthe systems and methods disclosed herein can be implemented according toan embodiment. FIG. 1 illustrates a typical basic deployment of acommunication system that includes macrocells, picocells, and enterprisefemtocells. In a typical deployment, the macrocells can transmit andreceive on one or many frequency channels that are separate from the oneor many frequency channels used by the small form factor (SFF) basestations (including picocells and enterprise or residential femtocells).In other embodiments, the macrocells and the SFF base stations can sharethe same frequency channels. Various combinations of geography andchannel availability can create a variety of interference scenarios thatcan impact the throughput of the communications system.

FIG. 1 illustrates an example of a typical picocell and enterprisefemtocell deployment in a communications network 100. Macro base station110 is connected to a core network 102 through a backhaul connection170. Subscriber stations 150(1) and 150(4), or user equipment (UE) usingLTE terminology, can connect to the network through macro base station110. In the network configuration illustrated in FIG. 1, office building120(1) causes a coverage shadow 104. Pico station 130, which isconnected to core network 102 via backhaul connection 170, can providecoverage to subscriber stations 150(2) and 150(5) in coverage shadow104.

In office building 120(2), enterprise femtocell 140 provides in-buildingcoverage to subscriber stations 150(3) and 150(6). Enterprise femtocell140 can connect to core network 102 via ISP network 101 by utilizingbroadband connection 160 provided by enterprise gateway 103.

FIG. 2 is a block diagram of another wireless communication network inwhich the system and methods disclosed herein is implemented accordingto an embodiment. FIG. 2 illustrates a typical basic deployment in acommunications network 200 that includes macrocells and residentialfemtocells deployed in a residential environment. Macrocell base station110 is connected to core network 102 through backhaul connection 170.Subscriber stations 150(1) and 150(4) can connect to the network throughmacro base station 110. Inside residences 220, residential femtocell 240can provide in-home coverage to subscriber stations 150(7) and 150(8).Residential femtocells 240 can connect to core network 102 via ISPnetwork 101 by utilizing broadband connection 260 provided by cablemodem or DSL modem 203.

Backhauls 170, 160, and 260, in both wireline and wireless forms, have amaximum and sometimes variable capacity for backhauling the traffic toand from a particular base station 110, 130, 140, or 240, and thereforedemand for backhaul resources may exceed capacity. This congestioneffect may occur for both wired and wireless network base stations andwhen using either wired or wireless backhaul technology.

FIG. 3 is a block diagram of a subset 100′ of the wireless communicationsystem of FIG. 1 in which a base station has multiple backhaul optionsavailable. In this embodiment, the base station is pico station 130which now has a backhaul connection 170 and an alternative backhauloption 170′. Both backhaul options connect the pico station to corenetwork 102. While FIG. 3 depicts the concept of multiple backhauls withrespect to pico station 130 and backhaul connection 170, one skilled inthe art will understand that the invention herein would apply for macrostation 110, as well. Similarly, enterprise femtocell 140 may havemultiple backhaul options such as a dialup line as an alternative to thebroadband connection 160 provided by enterprise gateway 103 or may havean alternate broadband connection provided by an alternate gateway.Residential femtocell 240 may also have multiple backhaul options suchas a dialup line as an alternative to the broadband connection 260provided by a cable modem or DSL modem 203 or may have an alternatebroadband connection provided by an alternate modem, for instance it maybe able to choose between the broadband connections provided by both acable modem and a DSL modem. Although depicted as a single PtPconnection, the term “backhaul” as used herein may also refer to acommunication path involving multiple hops, connected end-to-end in aserial fashion, each of which uses different technologies. For example,a backhaul connection may consist of a licensed-band wireless link plusan Ethernet over fiber link. Additionally, a base station may havemultiple backhaul connections of the same type. While FIG. 3 depicts abase station with two backhaul connections, other stations with multiplebackhauls may have more than two backhaul connections.

FIG. 4 is a functional block diagram of a station 300. In someembodiments, the station 300 is a wireless or wireline network node,such as a base station, an LTE eNB, a cable head end, or other networknode (e.g., the macro base station 110, pico station 130, enterprisefemtocell 140, enterprise gateway 103, or residential femtocell 240shown in FIGS. 1, 2, and 3). The station 300 comprises a user data planeprocessing module 320 and a control plane processing module 330 bothcommunicatively coupled to an access network transmitter receiver module(or transceiver) 310 and to a backhaul distribution module 340. Thebackhaul distribution module 340 is further communicatively coupled tobackhaul A 351 and backhaul B 352. In one embodiment, the modules instation 300 are coupled via control logic.

The transmitter receiver module 310 is configured to transmit andreceive communications with other devices such as subscriber stations150. In one embodiment, the communications are transmitted and receivedwirelessly. In another embodiment, the communications are transmittedand received over wire. In one embodiment, the transmitter receivermodule includes a modem, an antenna and a radio.

The user data plane processing module 320 is configured to process userdata communications being received and transmitted by the station 300.In an embodiment, the user data plane processing module 320 analyzesapplication classes and/or specific applications of the datacommunications being received and transmitted by the station 300.

The user data plane processing module 320 may inspect each packet todetect the application information including the application class andspecific application. Some example application classes are 2-way videoconferencing, unidirectional streaming video, online gaming, and voice.Specific applications refer to the particular software used to generatea data stream. Some example specific applications are Youtube, Netflix,Skype, and iChat. Each application class can have numerous specificapplications. In an embodiment, additional application information, suchas a video codec used, a frame rate, and a frame size, is also detectedfor some application types. Further, application information may includetransport information, for example, allowing the backhaul distributionmodule 340 to distinguish data for video transported using TCP fromvideo transported using UDP.

In an embodiment, the application class and specific application can bedetected by inspecting IP source and destination addresses. The userdata plane processing module 320 can perform a reverse domain namesystem (DNS) lookup or Internet WHOIS query to establish the domain nameand/or registered assignees for the IP addresses. The domain name and/orregistered assignee information can be used to establish bothapplication class and specific application for the data stream based ona priori knowledge of the domain or assignee's purpose. For example, iftraffic with a particular IP address yielded a reverse DNS lookup orWHOIS query that included the name ‘Youtube’ then this traffic streamcould be considered a unidirectional video stream (application class)using the Youtube service (specific application).

The application class and specific application can also be detected byinspecting an HTTP header. For example, the ‘Host’ field in the HTTPheader typically contains domain or assignee information which, asdescribed in the embodiment above, is used to map the stream to aparticular application class or specific application. For anotherexample, the ‘Content Type’ field in an HTTP packet can be inspected forinformation regarding the type of payload.

The application class and specific application can be detected for somepackets by inspecting other packets. For example, the user data planeprocessing module 320 may inspect a protocol sent in advance of the datastream. The application class or specific application may be detectedbased on the protocol used to set up or establish a data stream insteadof identifying this information using the protocol used to transport thedata stream. For example, Real Time Streaming Protocol (RTSP) packetscan be used to establish multimedia streaming sessions. Information sentfrom a client to a server to set up a multimedia streaming session canbe used when processing the streaming data sent from the server to theclient. Additionally, information about an RTSP streaming session, suchas the current playback time and requested range of playback times, canbe used to determine current bandwidth demand and predicted futurebandwidth demand.

Although the embodiment illustrated in FIG. 4 has a user data planeprocessing module 320 and a control plane processing module 330 that aredistinct, another embodiment may have a combined data processing module.Additionally, the detection of application information may be performedin a module other than the user data plane processing module 320, forexample, a packet inspection module. Example apparatuses and methods fordetecting application information are described in U.S. patentapplication Ser. No. 13/236,308, filed Sep. 19, 2011, which is herebyincorporated by reference.

The control plane processing module 330 is configured to process controlcommunications being received and transmitted by the station 300. Thesemay be control communications such as from an LTE Mobility ManagementEntity (MME) or Element Management System (EMS) directed at controllingthe actions of the station 300 or may be control communications fromentities such as an MME or EMS directed at controlling the actions ofdevices with which the base station communicates via access networktransmitter receiver module 310, such as subscriber stations 150.

The backhaul distribution module 340 determines which data plane andcontrol plane communications, if any, will be transported to the corenetwork 102 via backhaul A 351 or via backhaul B 352. The station 300includes backhaul interface modules to provide communications overbackhaul A and backhaul B. For concise description, transporting datavia one of the backhaul interface modules and the associated backhaulconnection may be referred to simply as using backhaul A or backhaul B.

In one embodiment, station 300 may be an LTE picocell capable oftransmitting wirelessly towards user stations at peak rates of 150 Mbpsand receiving from user stations at peak rates of 70 Mbps. Since theuplink cannot exceed a sustained rate of 70 Mbps, and variable channelconditions will likely keep the average sustained downlink data ratebelow 100 Mbps, it is reasonable to backhaul such a station with a 100Mbps full duplex Ethernet link. However, in addition to user data andcontrol data destined for user equipment 150, the backhaul in thedirection towards an LTE picocell may contain control data destined forthe picocell itself and not governed by the over the air limitation of150 Mbps. While this data is generally small, it can be large, forexample, when new software is downloaded to the picocell by an EMSentity for later use. Additionally, since data rates in the core networkcan exceed those over an LTE air link, the peak demand for bandwidth tosend data to the picocell may be greater than the 150 Mbps air linkrate. So, while 100 Mbps may be sufficient for average demand forbackhaul capacity towards the picocell, it may be insufficient to handlethe peak demand for backhaul bandwidth.

In the above embodiment, if peak demand for backhaul capacity exceedsthe capacity of the backhaul there may be three solutions. First,assuming the average demand for backhaul bandwidth is less than thecapacity of the backhaul, data may be delayed until demand subsides.However, this delay may be too long to preserve the quality of userexperience, especially for services such as voice and video, and it maytake a very long time for the demand to reduce to less than the linkcapacity if the average backhaul demand is normally close to linkcapacity. Second, the original 100 Mbps Ethernet link could be replaced(either at installation, by design, or after the fact, by necessity)with a gigabit Ethernet link. However, a gigabit Ethernet link may bemuch more expensive to install and operate and provides more than sixtimes the necessary peak capacity. A third option is to provide a secondbackhaul, backhaul B 352, that is sized to address some or all of theexcess backhaul capacity needs.

In an embodiment that contains multiple backhauls, backhaul distributionmodule 340 determines when to use backhaul A and when to use backhaul Band determines which data should be transported over backhaul A andwhich data should be transported over backhaul B. In combination, thesedecisions also determine how much data is transported over backhaul Aand how much data is transported over backhaul B. These decisions may bein response to backhaul congestion, i.e. backhaul demand exceedingbackhaul capacity, may be in response to anticipated backhaulcongestion, or may be based on an ongoing policy that always placescertain data on backhaul A while always placing certain other data onbackhaul B. The methods by which data is directed on one backhaul oranother may vary depending upon whether the base station is receivingthe data on the backhaul or transmitting data on the backhaul.

There are many factors that may make one backhaul, backhaul A 351, moreor less attractive than another backhaul, backhaul B 352, for some, all,or certain data. In one embodiment, backhaul A may have a differentfinancial cost associated with the transport of a specific quantity ofdata than backhaul B. For instance, backhaul A may be owned and operatedby the owner of station 300 while backhaul B may be owned and operatedby another entity that charges based on usage, e.g., the number of bytestransported. When one backhaul has a higher financial cost associatedwith use, it would be desirable to minimize the use of the higher costbackhaul.

In another embodiment, backhaul A may have different delay than backhaulB. For instance if backhaul B were a satellite link and backhaul A werenot, backhaul B may have substantially more delay and would be lessdesirable. Similarly, in an embodiment where backhaul B is a shared linkand backhaul A is a dedicated link, backhaul B may both have more delayand less deterministic throughput making it less desirable than backhaulA. Detected application information may additionally be used in decidingwhat data to communicate on backhaul A and what data to communicate onbackhaul B. For example, data for a video streaming application maypreferentially use a backhaul with lower delay.

If, in an embodiment, backhaul B were implemented using a technologythat used unlicensed spectrum, such as an unlicensed spectrumpoint-to-point link or an unlicensed spectrum point-to-multipoint ormesh technology such as 802.11a WiFi, and backhaul A used a wireline orlicensed spectrum technology, backhaul B may be less robust tointerference and errors and therefore less desirable than backhaul A.

In another embodiment, backhaul A may require different computationalresources than backhaul B. For instance, backhaul A may be owned andoperated by the owner of station 300 while backhaul B may be owned andoperated by another entity. In this scenario, backhaul B may beuntrusted and may require the application of additional security, suchas IP-SEC. Such additional security may require substantial resources.The station 300 may only have sufficient resources to apply theadditional security to a fraction of the data transported over thebackhaul. Additionally, the need for additional computational resourcesmay cause the station 300 to consume more electrical power, increasingoperational expenses and reducing the mean time to failure of thedevice.

FIG. 5 is a functional block diagram of an LTE picocell 500 that is anembodiment of station 300 of FIG. 4. Picocell 500 is mounted on cablestrand 501. Cable strand 501 provides physical support to the LTEpicocell and may also provide power. Alternatively, power can beprovided from another source, such as solar power or batteries. Cablestrand 501 also provides communication path 561 which is accessible viacable modem technology. Picocell 500 contains two backhaul options.Cable Modem backhaul 551 provides backhaul over communication path 561and ultimately over the cable strand 501 using cable modem technologysuch as the well-known Data-Over-Cable Service Interface Specification(DOCSIS) standard. 802.11a backhaul 552 provides wireless backhaul usingtechnology based on the Institute of Electrical and ElectronicsEngineers (IEEE) Wireless Local Area Network (LAN) specifications andmay use point-to-point, point-to-multipoint, or mesh variants of theIEEE 802 technology. 802.11a backhaul 552 uses one or more antennas 562to transmit and receive backhaul data.

Backhaul distribution module 340 determines which services or data aretransported over the cable modem backhaul or the 802.11a backhaul. Cablemodem backhaul 551 may be the preferred backhaul for LTE picocell 500.The cable modem backhaul may provide more robust and reliable transportof data than 802.11a backhaul 552 which operates using unlicensedspectrum and generally follows a listen-before-talk protocol whichallows for data collisions. Additionally, if the picocell is operated bythe cable operator in control of cable strand 501, transport of datausing the cable modem backhaul may be more trusted than the 802.11abackhaul and, therefore, require less security to be implemented.Alternatively, 802.11a backhaul may be preferred if the picocelloperator is not the cable strand operator. In this case, the trust levelmay be the same and there may be a lower financial cost associated withtransporting data on the 802.11a backhaul compared to the cable modembackhaul.

FIG. 6 is a flow diagram for a method for determining when to use analternate backhaul in an embodiment where backhaul A is alwayspreferable to backhaul B. Even though the flow diagram of FIG. 6 showssuch an embodiment, alternate embodiments exist where backhaul A may bepreferable for certain data while backhaul B is preferable for otherdata. The logic of the flow diagram of FIG. 6 may be implemented, forexample, in backhaul distribution module 340 of station 300. Flow startsat step 410, where the station is monitoring the demand for bandwidth onthe backhaul. The monitoring can be implemented by numerous methods. Inone embodiment, the monitoring is implemented by summing the guaranteedbit rates or the maximum bit rates of services that are currently activeor some function of these bit rates. In an alternative embodiment, themonitoring is performed by measuring traffic on the backhaul andpredicting future bandwidth demand based on previous bandwidth demand.The monitoring can be done real time on a per packet basis. It can alsobe performed on a fixed or variable time interval basis. If the backhauluses a technology that may provide variable bandwidth, such as awireless link implementing adaptive modulation and coding, themonitoring step 410 can be enhanced to also monitor the capacity of thebackhaul in addition to the demand for backhaul bandwidth. Allowance maybe made for extra bandwidth demand created by service relatedretransmissions over the backhaul such as those caused by an automaticretry request (ARQ) mechanism as can be found in transmission protocolssuch as the well-known transmission control protocol (TCP). Allowancemay also be made for backhaul bandwidth availability reduction due tobackhaul transmission technology ARQ or hybrid-ARQ (HARQ).

At decision block 420, a determination is made whether demand forbackhaul bandwidth exceeds the capacity of backhaul A 351. In oneembodiment, demand is determined to exceed capacity if demand exceedscapacity for any amount of time however short, to avoid any delay due todata waiting to be transmitted over the backhaul. In an alternativeembodiment, some amount of delay may be tolerated, in which case demandfor backhaul bandwidth is deemed to exceed backhaul capacity if thedemand exceeds the capacity by a specific amount of bandwidth for atleast a certain amount of time. If at step 420, it is determined thatdemand does not exceed capacity, flow returns to step 410 wheremonitoring is continued. If, however, at step 420 it is determined thatdemand does exceed the capacity of backhaul A, flow proceeds to step 440where some data is distributed onto backhaul B 352.

From step 440, flow continues to step 450 where the station monitorsdemand for backhaul bandwidth and, depending on the backhaul technology,monitors backhaul capacity. Step 450 is similar to step 410, except insome embodiments it is preferable to not only monitor for situationswhere data or services must be offloaded to backhaul B, but also monitorfor situations where data or services may be returned to backhaul A.

At decision point 460, a determination is made whether there is sparecapacity on backhaul A. If there is spare capacity on backhaul A, flowproceeds to step 480 where data or services are returned to backhaul A.From step 480, flow returns back to step 410 to monitor backhaul demand.Alternatively, if some but not all data or services were returned tobackhaul A, flow could proceed back to step 450 instead of step 410. Ifat step 460 there was no spare capacity on backhaul A, flow proceeds todecision point 470 where a determination is made whether backhaul A isstill oversubscribed such that demand for bandwidth by the services anddata still transported over backhaul A still exceeds the capacity ofbackhaul A. If at step 470 it is determined that backhaul A is stilloversubscribed, flow returns to step 440 to distribute more data orservices onto backhaul B. If at step 470 it is determined that backhaulA is not oversubscribed, flow returns to step 450 where the bandwidthdemands and backhaul capacity are monitored.

When data is distributed onto an alternate backhaul, as in step 440 ofFIG. 6, the decision of what data to transport over the alternatebackhaul can have many factors. One factor may be the type of data. Thetype of data may be based on detecting information about applicationsassociated with the data. For instance, real-time data such as voice orvideo may be preferentially transported over the lower latency backhauloption, allowing email, file transfer, and other services less sensitiveto latency to be carried over a higher latency backhaul option.Similarly, data that is error tolerant, such as data transmitted with atransport protocol such as TCP that incorporates an ARQ function may betransmitted over a less reliable unlicensed spectrum wireless backhaulwhile data that cannot tolerate errors or cannot tolerate the delaycaused by retransmissions may be transported on a less error-pronebackhaul alternative such as wired Ethernet or a licensed spectrumwireless backhaul.

The choice of which data to distribute onto alternate backhauls may alsobe impacted by the technology deployed by station 300. For instance, ifstation 300 is a pico station implementing LTE technology, data to andfrom a specific piece of user equipment is segregated onto one or morelogical bearers between a data gateway and the user equipment. The picostation may have little visibility into the data within the bearer butmay know information such as guaranteed or maximum bit rates. In thisembodiment, the pico station can cause the bearers to be set updistributed across alternative backhauls on a bearer by bearer basiswith all data transported in an individual bearer transported on thesame backhaul option. In an alternate embodiment, the decision todistribute data traffic across alternate backhaul options could occur ona user equipment basis with all data for an individual user equipmentinstance transported on the same backhaul option.

For example, each end user device may be assigned a unique InternetProtocol (IP) address. IP routing functions within the pico station (forexample, implemented in backhaul distribution module 340) and gatewaydevices may be used to distribute uplink traffic streams acrossalternate backhaul options based upon the user IP address basis. Therouting decision may be based upon the minimization of a cost functionwhich continuously gathers the environmental parameters described hereinsuch as available capacity, tariff, transport reliability or othermeasures. These environmental parameters may be collected locally by thedevice itself or by receiving signaling messages sent by a device on theother side of the backhaul. For example, the core network gateway 102may collect information on the current state of congestion across links170 and 170′ using internal methods as well as collect congestioninformation signaled from pico station 130. One skilled in the art wouldrecognize that in addition to distribution based upon IP addresses,distribution may be based on IP sockets, port numbers or Ethernet MediaAccess Control (MAC) addresses.

Other embodiments may use technology that would allow packet by packetdistribution of data on alternate backhauls. This may be accompanied bypacket inspection that allows data to be distributed across backhauloptions based on service type or application type.

Once a distribution choice has been made, different embodiments may havemore or less ability to redistribute data. For instance, it may bedifficult to move an LTE bearer from one backhaul to another, so it maybe preferable to make the decision to distribute a bearer onto analternative backhaul at bearer creation and to not move it during thelife of the bearer at a pico station. However, if the gateway allows,the path a bearer takes between the gateway and the pico station may bechanged to route the bearer and its data over a different backhaul.Similarly, in some embodiments, the choice of backhaul option to supportan individual user equipment instance may be made at the point when theuser equipment enters the network or connects with the pico station.

There are many ways to redistribute all or part of data to differentbackhauls depending on the network architecture of the base station. Forthe most common case where backhaul data is transported as IP packetpayload across different backhaul IP networks, as long as the backhauldata IP packets reach the intended backhaul destination (e.g. MME/SGW ina LTE network), the redistribution of data by the base station todifferent backhaul networks is transparent to the backhaul destination.

A base station with multiple backhaul networks is a multi-homed nodeattached to multiple computer networks. Standard and proprietary routingprotocols can be used for the multi-homed base station to notify therouters on the connected backhaul networks how packets destined for theLTE downlink should be delivered to the base station. In an embodiment,backhaul distribution module 340 can signal one or more routers in corenetwork 102 with updates to their static routing tables therebyadjusting the flow of downlink packets based on end-user IP address.Updates may be communicated using standard protocols such as SimpleNetwork Management Protocol (SNMP), Hyper-Text Transport Protocol(HTTP), Telnet or Secure Shell (SSH). Alternatively, updates may becommunicated via a proprietary protocol. In an alternative embodiment,backhaul distribution module 340 can send Border Gateway Protocol (BGP)route update messages to the routers in core network 102. The routeupdate messages can contain different LOCAL_PREF attributes fordifferent backhaul networks according to the base station's preference.The neighbor routers on the backhaul networks will route the packets tothe backhaul network with the highest LOCAL_PREF value. The base stationcan also send route update messages with different AS_PATH attributesfor different backhaul networks, and the neighbor routers on thebackhaul networks will route packets to the backhaul network with theshortest AS_PATH. The base station can also send route update messagescontaining Multi-Exit Discriminator (MED) values for different backhaulnetworks, and the neighbor routers on the backhaul networks will routethe packets to the backhaul network with lower MED values.

Proprietary routing protocols can also be used between the base stationand the routers on multiple backhaul networks. For example, aproprietary protocol can be a protocol similar to BGP but with supportof per UE tunnel routing. A base station can send route update messagesto neighbor routers containing different weights for different tunnelson different backhaul networks. Alternatively, a base station can sendroute update messages to neighbor routers containing static routeinformation for different tunnels. A router that implements theproprietary protocol can route packets for different UE tunnels todifferent backhaul networks.

There are multiple ways to configure the base station's uplink routingengine to distribute all or part of data IP packets to a differentbackhaul network. For a base station with multiple backhaul IP networks,the simplest approach is to change the default route of the base stationrouting engine to the desired backhaul network, for instance the onewith lowest cost, and effectively use only the selected backhaul networkas the default route. This simple approach is quite effective for afixed point to multiple point WiMAX network where different subscriberstations are assigned different IP addresses and the WiMAX base stationserves as a router between the subscriber's IP network and the backhaulIP networks. A slightly more advanced approach could be assigningdifferent weights to different backhaul networks, and the routing enginein the base station can distribute data across all the backhaul networksaccording to the weight and favor some backhaul networks over others.

For base stations that can provide different QoS to different users anddifferent types of traffic, a more advanced routing engine that supportdynamic routing on a per packet basis can be used to distribute trafficfrom different users or different type of services to different backhaulnetworks. For example, in an LTE pico station, IP packets from differentbearers can be inspected and marked with different tags according to thetunnel ID of the eGTP-U tunnels between different UEs and the MME/SGW onthe core network. In a WiMax base station, IP packets from differentsubscribers and service flows can be inspected and marked with differenttags according to service flow identifiers. Once packets are marked withdifferent tags, the routing engine can route the IP packets to differentbackhaul networks according to the tags associated with the IP packetson a per packet basis.

Those of skill will appreciate that the various illustrative logicalblocks, modules, controllers, and algorithm steps described inconnection with the embodiments disclosed herein can be implemented aselectronic hardware, computer software, or combinations of both. Toclearly illustrate this interchangeability of hardware and software,various illustrative components, controllers, blocks, modules, and stepshave been described above generally in terms of their functionality.Whether such functionality is implemented as hardware or softwaredepends upon the particular system and design constraints imposed on theoverall system. Skilled persons can implement the describedfunctionality in varying ways for each particular application, but suchimplementation decisions should not be interpreted as causing adeparture from the scope of the invention. In addition, the grouping offunctions within a module, block or step is for ease of description.Specific functions or steps can be moved from one unit, module or blockwithout departing from the invention.

The various illustrative logical blocks, controllers, and modulesdescribed in connection with the embodiments disclosed herein can beimplemented or performed with a general purpose processor, a digitalsignal processor (DSP), an application specific integrated circuit(ASIC), a field programmable gate array (FPGA) or other programmablelogic device, discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein. A general-purpose processor can be a microprocessor,but in the alternative, the processor can be any processor, controller,microcontroller, or state machine. A processor can also be implementedas a combination of computing devices, for example, a combination of aDSP and a microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

The steps of a method or algorithm and the processes of a block ormodule described in connection with the embodiments disclosed herein canbe embodied directly in hardware, in a software module executed by aprocessor, or in a combination of the two. A software module can residein RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory,registers, hard disk, a removable disk, a CD-ROM, or any other form ofmachine or computer readable storage medium. An exemplary storage mediumcan be coupled to the processor such that the processor can readinformation from, and write information to, the storage medium. In thealternative, the storage medium can be integral to the processor. Theprocessor and the storage medium can reside in an ASIC.

The above description of the disclosed embodiments is provided to enableany person skilled in the art to make or use the invention. Variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the generic principles described herein can beapplied to other embodiments without departing from the spirit or scopeof the invention. Thus, it is to be understood that the description anddrawings presented herein represent a presently preferred embodiment ofthe invention and are therefore representative of the subject matter,which is broadly contemplated by the present invention. It is furtherunderstood that the scope of the present invention fully encompassesother embodiments that may become obvious to those skilled in the artand that the scope of the present invention is accordingly limited bynothing other than the appended claims.

1. A communication station, comprising: a transceiver arranged fortransmitting and receiving data with subscriber stations; a plurality ofbackhaul interface modules, each backhaul interface module beingarranged for providing backhaul communications over a corresponding oneof a plurality of backhaul connections; a data processing module fordetecting information about applications associated with the datatransmitted and received by the transceiver; and a backhaul distributionmodule coupled to the plurality of backhaul interface modules andarranged for monitoring demand for backhaul bandwidth and distributingdata over the plurality of backhaul connections based on the demand forbackhaul bandwidth and the detected information about the applicationsassociated with the data.
 2. The communication station of claim 1,wherein the data processing module comprises: a user data planeprocessing module coupled to the backhaul distribution module and thetransceiver and arranged for processing user data communications beingreceived and transmitted by the communication station; and a controlplane processing module coupled to the backhaul distribution module andthe transceiver and arranged for processing control communicationsreceived and transmitted by the communication station, and whereindistributing data over the plurality of backhaul connections by thebackhaul distribution module includes whether the data is associatedwith the user data plane or the control plane.
 3. The communicationstation of claim 1, wherein the backhaul distribution module is furtherarranged for monitoring capacity of the plurality of backhaulconnections, and wherein distributing data over the plurality ofbackhaul connections by the backhaul distribution module includes thecapacity of the plurality of backhaul connections.
 4. The communicationstation of claim 1, wherein the detected information about theapplications associated with the data comprises application classes andspecific applications.
 5. The communication station of claim 1, whereinthe demand for backhaul bandwidth is based on the detected informationabout the applications associated with the data.
 6. The communicationstation of claim 1, wherein the demand for backhaul bandwidth monitoredby the backhaul distribution module is based on guaranteed bit ratesassociated with transmitting and receiving data with the subscriberstations.
 7. The communication station of claim 1, wherein the demandfor backhaul bandwidth monitored by the backhaul distribution module isbased on maximum bit rates associated with transmitting data to andreceiving data from the subscriber stations.
 8. The communicationstation of claim 1, wherein the backhaul distribution module distributesdata over the plurality of backhaul connections by: initiallydistributing data to a preferred one of the plurality of backhaulinterface modules; determining whether the demand for backhaul bandwidthexceeds the capacity of the backhaul connection associated with thepreferred one of the backhaul interface modules; and distributing datato an alternate one of the plurality of backhaul interface modules whenthe demand for backhaul bandwidth exceeds the capacity of the backhaulconnection associated with the preferred one of the backhaul interfacemodules.
 9. The communication station of claim 8, wherein the backhauldistribution module further distributes data over the plurality ofbackhaul connections by: after distributing data to the alternate one ofthe plurality of backhaul interface modules, determining whether sparebackhaul bandwidth exists on the backhaul connection associated with thepreferred one of the backhaul interface modules; and redistributing datafrom the alternate one of the plurality of backhaul interface modules tothe preferred one of the backhaul interface modules when spare backhaulbandwidth exists on the backhaul connection associated with thepreferred one of the backhaul interface modules.
 10. A method for usewith a communication station operable to communicate with userequipments via a transceiver and communicate with a network over aplurality of backhaul connections, the method comprising: monitoringdemand for backhaul bandwidth; distributing backhaul data to a preferredone of the plurality of backhaul connections; detecting informationabout applications associated with the backhaul data; determiningwhether the demand for backhaul bandwidth exceeds the capacity of thepreferred one of the plurality of backhaul connections backhaul; anddistributing at least some of the backhaul data to an alternate one ofthe plurality of backhaul connections when the demand for backhaulbandwidth exceeds the capacity of the preferred one of the plurality ofbackhaul connections, the backhaul data distributed to an alternate oneof the plurality of backhaul connections being selected utilizing thedetected information about the applications associated with the backhauldata.
 11. The method of claim 10, further comprising: after distributingat least some of the backhaul data to the alternate one of the pluralityof backhaul connections, determining whether spare backhaul bandwidthexists on the preferred one of the plurality of backhaul connections;and redistributing at least some of the backhaul data from the alternateone of the plurality of backhaul connections to the preferred one of theplurality of backhaul connections when spare backhaul bandwidth existson the preferred one of the plurality of backhaul connections.
 12. Themethod of claim 10, wherein the detected information about theapplications associated with the backhaul data comprises applicationclasses and specific applications.
 13. The method of claim 10, whereinthe at least some of the backhaul data distributed to the alternate oneof the plurality of backhaul connections is selected to minimize a costfunction associated with communications on the plurality of backhaulconnections.
 14. The method of claim 13, wherein the cost function isbased at least in part on attributes of the backhaul connectionsselected from the group consisting of financial costs, communicationdelays, communication robustness, and computational resources of thecommunication station associated with communicating data over thebackhaul connections.
 15. The method of claim 10, wherein determiningwhether the demand for backhaul bandwidth exceeds the capacity of thebackhaul connection associated with the preferred one of the pluralityof backhaul connections comprises determining whether the demand forbackhaul bandwidth exceeds the capacity of the preferred one of theplurality of backhaul connections by a threshold amount and for athreshold time.
 16. The method of claim 10, wherein distributing atleast some of the backhaul data to the alternate one of the plurality ofbackhaul connections comprises distributing backhaul data associatedwith a bearer.
 17. The method of claim 10, wherein distributing at leastsome of the backhaul data to the alternate one of the plurality ofbackhaul connections comprises distributing backhaul data associatedwith one of the user equipments.
 18. The method of claim 10, whereinmonitoring demand for backhaul bandwidth comprises utilizing guaranteedbit rates associated with communicating with the user equipments. 19.The method of claim 10, wherein monitoring demand for backhaul bandwidthcomprises utilizing maximum bit rates associated with communicating withthe user equipments.